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Abstract: This paper focuses on the recent escalation in application of CFD to manned and unmanned flight 
projects at NASA and the need to often apply these methods to problems for which little or no previous 
validation data directly applies. The paper discusses the evolution of NASA’s CFD development from a strict 
“Develop, Validate, Apply” strategy to sometimes allowing for a “Develop, Apply, Validate” approach. The risks 
of this approach and some of its unforeseen benefits are discussed and tied to specific operational examples. 
There are distinct advantages for the CFD developer that is able to operate in this paradigm, and 
recommendations are provided for those inclined and willing to work in this environment. 


INTRODUCTION 

Traditionally, Computational Fluid Dynamics (CFD) 
methods developers have applied a very logical, 
systematic approach to software development, termed 
here as the ‘Develop, Validate, Apply” strategy. In 
this strategy, “Develop” refers to the coding and 
verification of the CFD software, while “Validate” 
refers to the process of running test cases on the 
software and comparing with known data. These 
validation data sources can be from other validated 
CFD methods, sub-scale experiments such as wind 
tunnel tests, or in rare cases, full-scale data, such as that 
obtained from flight tests. Upon completion of the 
validation phase of the strategy, the validated software 
system moves into the “Apply” stage and is delivered 
to the end-users who apply the method to problems that 
fit within the range of validation established for the 
method. This conservative strategy for CFD 
development ensures that the users of the method have 
a fully tested and validated method for their given 
application and as long as they don’t stray too far from 
the parameters under which the method was validated, 
they have a tool that is capable of predicting flow 
characteristics within a demonstrated error band. 
However, as most anyone who develops CFD methods 
already knows, end-user applications can quickly 
migrate away from the parameters under which the 
software was validated. 

In some cases, it becomes necessary for end-users to 
apply the methods to problems that are well outside the 
validation range of the software. Depending on the 
criticality of the pending analysis, the users may be 
forced into a situation where the final stages of the 
preferred strategy become switched and we find 
ourselves operating in the less systematic framework of 


“Develop, Apply, Validate.” This places the end-user 
in a situation where they have little or no knowledge of 
neither how the method will operate for their particular 
problem nor how accurate it will be. As a result, large 
modeling uncertainties are added to these types of 
analyses to account for this unknown performance. 
These additional uncertainties can have a profound 
impact on the prediction of performance and the overall 
design of a system. This paper discusses how one 
might be required to operate under this strategy and the 
ramifications of implementing this software maturation 
strategy. 

NASA’S EVOLVING CFD MATURATION 
PROCESS 

In the past seven years, the National Aeronautics and 
Space Administration (NASA) has seen a marked 
change in how they apply and ultimately develop and 
validate their Computational Fluid Dynamics (CFD) 
methods. This change comes as a result of two major 
Agency events. The first is the loss of the Space 
Shuttle Columbia during reentry in 2003. The 
second is the initiation of NASA’s new exploration 
vision, embodied in the Agency’s Constellation 
Program, which requires NASA engineer’s to be 
responsible for large quantities of the aeroscience data 
products associated with the program’s crewed 
spacecraft and launch vehicle. These two events 
coalesced to fundamentally change how the Agency 
applies its CFD methods, develops new CFD capability, 
and validates this capability. 

CFD and the Space Shuttle Program 

On January 16, 2003 NASA launched the Space 
Shuttle Columbia on what would turn out to be its 28 th 



and final mission. Roughly 82 seconds into the flight, a 
piece of insulating foam from the left bipod ramp on 
the vehicle’s External Tank (ET) struck the left wing 
leading edge of Columbia, critically damaging the 
vehicle. Upon reentry, damage from the debris strike 
resulted in very high temperature flow reaching the 
vehicle’s primary structure causing catastrophic 
structural failure and vehicle breakup over the 
Southwest United States. Following the accident, a 
formal investigation was conducted by a board of 
experts commissioned by the President of the United 
States. This Columbia Accident Investigation Board 
(CAIB) sponsored tests and analyses to determine the 
root cause of the accident and published their findings 
in a report commonly known as the CAIB Report 1 . 
Among the causes contributing to the accident, NASA’s 
Organization and Safety Culture were identified as key 
contributors as “Shuttle Program management made 
erroneous assumptions about the robustness of a system 
based on prior success rather than on dependable 
engineering data and rigorous testing.” 

In partial response to the accident and the findings 
of the investigation, NASA established a new 
organization known as the NASA Engineering and 
Safety Center (NESC). The NESC operates under the 
philosophy of three primary tenets to ensure safety: 

1) Strong in-line technical checks and balances to 
ensure proper engineering analysis and data are 
being applied to the problem. 

2) Healthy tension between the Project, Safety, and 
Engineering components supporting the 
Program. 

3) “Value added” independent assessment of 
problems that cannot be adequately resolved 
within the internal Program environment. 

The NESC is funded through NASA’s Office of the 
Chief Engineer, which allows it to maintain 
independence from the programs and projects it is 
asked to support. The independent assessment 
functionality of the NESC is one of the unique features 
of the organization that allows it to provide a technical 
evaluation of a given situation without biases due to 
schedule and budgetary pressures. The NESC 
presently supports 15 discipline-centric teams, each 
made up of experts from across NASA, other 
government agencies, industry, and academia. CFD is a 
key analysis tool of the Aerosciences Technical 
Discipline Team (TDT) as well as several other TDTs 
within the NESC. 

The accident investigation resulting from the 
Columbia tragedy, as well as the highly increased 
technical insight into the Space Shuttle Program after it 
returned to flight status have stretched the Agency’s 
CFD resources and capabilities. The fast-paced 
environment of the accident investigation required 
rapid turn-around of analyses that used our CFD 
methods on a scale that had not been anticipated during 
their development. This forced engineers to use the 


codes in innovative applications, sometimes with 
limited validation applicable to the specific problems 
they were analyzing. Since the accident, NASA has 
successfully flown the Space Shuttle 19 times through 
May, 2010, see Table 1, and the use of CFD to analyze 
and evaluate problems on the vehicle has continued to 
escalate and expand at a very high rate. Examples of 
CFD usage on the shuttle will be provided for the five 
bolded flights shown in the table. 

Table 1. Space Shuttle Missions Since Columbia 


Accident 


Mission 

Launch 

Date 

Mission 

Launch 

Date 

STS-114 

7/26/2005 

STS-126 

11/14/2008 

STS-121 

7/4/2006 

STS-119 

3/15/2009 

STS-115 

9/9/2006 

STS-125 

5/11/2009 

STS-116 

12/9/2006 

STS-127 

7/15/2009 

STS-117 

6/8/2007 

STS-128 

8/28/2009 

STS-118 

8/8/2007 

STS-129 

11/16/2009 

STS-120 

10/23/2007 

STS-130 

2/8/2010 

STS-122 

2/7/2008 

STS-131 

4/5/2010 

STS-123 

3/15/2008 

STS-132 

5/14/201 0 

STS-124 

5/31/2008 




Far and away, the most dominant use of CFD on 
the shuttle since the Columbia accident has been for the 
prediction of launch debris and its transport. Prior to 
the Columbia accident, engineers and managers 
recognized the importance of predicting the transport of 
foam and ice debris during the ascent phase of the 
shuttle flight. Tools to predict debris transport were in 
development prior to the Columbia accident. Early 
debris prediction techniques used wind tunnel data to 
provide the aerodynamic environments necessary to 
make the predictions. In the late 1980’s, CFD began 
to make inroads in predicting these environments 2 , 
albeit on simplified geometry configurations. By the 
mid- 1 990 ’s, high-fidelity CFD analyses on much larger, 
more complex configurations were being used to 
predict shuttle aerodynamic environments 3 ' 4 . These 
simulations matured quickly and generally produced 
accurate surface pressure predictions when compared 
with experimental data as shown in Figure l 5 . 

During this period debris trajectory prediction 
analyses were performed on a case-by-case basis and a 
high degree of automation of this process was not 
available. In addition, the aerodynamic characteristics 
of the debris itself were not simulated. The 
investigation of the Columbia accident necessitated the 
rapid development of an automated debris transport 
analysis capability. In addition, the CAIB also 
recommended that the complete debris environment 
encountered by the Space Shuttle Launch Vehicle 
during ascent be characterized prior to Return-to-Flight 
(RTF). Since detailed aerodynamic characteristics of 
the debris components were not included in previous 




- 0.5 



Figure 1 . High-fidelity Space Shuttle CFD comparison 
with wind tunnel data (cf. Reference 5). 


debris transport analyses, these characteristics had to be 
developed in parallel with the RTF analyses. Various 
debris shapes were characterized using CFD and these 
data were validated using ballistic range test results as 
shown in Figure 2 6 . Thus the engineers found 
themselves validating their methodology virtually in 
parallel with the analyses being used to prepare the 
vehicle for its return to flight. This started to blur the 
line between apply and validate in the traditional 
process of CFD method maturation. Once the Space 
Shuttle returned to operation, this line became 
increasingly transparent and in some cases, the 
validate/apply order became completely reversed. 

STS-114 - Space Shuttle Returns to Flight 

On July 26, 2005, over two years after the Columbia 
accident, NASA launched STS-114 marking the Space 
Shuttle’s return to flight. Given the high sensitivity to 
debris and the uncertainty in many of the analyses 
employed leading up to this mission, numerous 
instruments and cameras were added to the flight 
manifest. In addition, maneuvers such as the “flip” 
maneuver of Figure 3, allowed the vehicle to be 
photographed and inspected to a level of detail 
unprecedented in prior flights. As a result of these 
inspections, a protruding gap filler was discovered 
between the ceramic tiles on the heatshield near the 
nose of the spacecraft. From the photographs, 
dimensions and shape of the protrusion could be 
derived and used as boundary conditions for the various 
boundary layer transition and heating analysis tools. 
These analyses, some involving CFD, were conducted 
over a span of just a few days while the vehicle was on 
orbit. The analyses showed the vehicle to be safe for 
return with the protruding gap filler, but uncertainties in 
the data produced sufficient concern for the safe return 
of the vehicle that mission managers formulated and 
exercised an Extra Vehicular Activity (EVA) to remove 
the offending component. Astronaut Steven Robinson 
was attached to the end of the arm of the Space Shuttle 
Orbiter’s Remote Manipulator System and maneuvered 
to the front of the vehicle to remove the gap filler, as 
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Figure 2. Comparison of aerodynamic characteristics 
of Space Shuttle debris computed by CFD 
with ballistic range data. (cf. Reference 6) 



Figure 3. Space Shuttle performing flip maneuver as it 
approaches the International Space Station. 





seen in Figure 4. This high-risk EVA underscored the 
importance of accurate analysis capability to future 
missions and the impact it could have on the risk 
posture of these missions. The gap filler analysis was 
also a harbinger of how our analysis techniques would 
be stressed and extended during future flights. 



Figure 4. Removal of protruding heatshield gap filler 


during STS- 114 mission. 

During STS- 114, numerous foam debris events were 
observed during the vehicle’s ascent indicating that the 
foam debris issue was not as well understood as 
presumed prior to the flight. As a result, the program 
waited nearly a year to conduct its next flight. During 
this time, several large sources of foam debris were 
removed from the ET, including two large Protuberance 
Air Loads (PAL) Ramps. These ramps were part of 
the original ET design to protect externally mounted 
cable trays from high velocity crossflows generated 
during ascent. Considerable testing and analysis, 
including CFD, were performed prior to STS- 114 to 
demonstrate that the cable trays remained structurally 
sound without the protection of the PAL ramps. The 
CFD involved unsteady separated flow over bluff 
bodies near a ground plane. This complex geometry 
and flow condition certainly stretched our CFD 
capability beyond the limits of its validation. 
Qualification testing to demonstrate that the cable trays 
could withstand the aerodynamic loads of a space 
Shuttle launch included unsteady pressure and 
structural response data that could be used to partially 
validate the CFD analysis. But again these data were 
being acquired in parallel with the process being 
exercised to qualify the system for flight. Thus the 
validation and application of the CFD were being 
performed simultaneously. 

The PAL ramps were identified as a potential debris 
source prior to STS- 114, but the uncertainty of the 
analysis, testing, and the risk of unintended 
consequences resulting from removal of the PAL ramps 
outweighed the risk of a foam strike originating from 
one of the ramps. During the STS-114 mission 
though, a large portion of one of the PAL ramps was 
lost during ascent, Figure 5. 



Figure 5. STS-114 PAL Ramp damage sustained during 


ascent. 

This documented foam loss elevated the risk of the PAL 
ramps as a foam source, and they were subsequently 
removed from the ET for the second return to flight 
mission, STS-121, and subsequently for all future 
missions. 

STS-121 Returns Debris Transport Prediction 
Validation Data 

Due to the large volume of data acquired during 
STS-114 and the resulting analyses, testing, and system 
changes precipitated by this flight, STS-121 was 
launched nearly a year after STS-114. Imagery data 
acquired during STS-121 demonstrated that the number 
of foam liberation events had been significantly 
reduced and the mission proceeded with few flight 
issues. The relatively clean flight provided engineers 
with an opportunity to perform more detailed analyses 
of some of the data that had been collected on both 
STS-114 and STS-121. In particular, some of the 
debris transport methodology that had undergone such 
rapid development could finally be compared against 
flight data 7 . Figure 6 shows two images of foam 
debris captured during the STS-121 launch that was 
liberated from one of the Space Shuttle ET ice/frost 
ramps. The debris travels aft and passes near the 
starboard Solid Rocket Booster (SRB) nozzle. The 
video imagery, in this case from a camera mounted near 
the nose of the SRB, is of sufficient quality and 
resolution to roughly estimate the size and trajectory of 
the foam debris. Figure 7 shows a predicted trajectory 
for a piece of foam released from the same ice/frost 
ramp at this point in the ascent trajectory with the 
debris ultimately passing right over the top of the SRB 
as observed in flight. The ability to correlate 
computational methods against full-scale flight data in 
this manner is often very difficult to accomplish in the 
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Figure 6. STS- 121 ascent imagery of foam debris 
liberated from the Space Shuttle ET. (cf. 
Reference 7). 



Figure 7. Predicted debris path (red line) for foam 
debris liberated from a specific ET ice frost 
ramp (cf. Reference 7). 


standard “Develop, Validate, Apply” maturation 
strategy due to the cost of conducting a flight test solely 
for the purpose of method validation. However, when 
the application and validation tasks are performed in a 
more parallel fashion as with the foam debris trajectory 
prediction, full-scale validation opportunities are often 
more forthcoming, particularly if the method 
developers are tightly integrated into the application 
and the definition of the flight measurements that could 
be used for validation. 


STS-118 - Tile Damage Due to Debris 

During the post launch inspection of STS-118, 
significant heatshield tile damage was observed on a 
pair of tiles on the starboard wing as shown in Figure 8. 
Digital photographs taken from aboard the International 
Space Station (ISS) as the shuttle approached and 
performed its flip maneuver showed that the debris had 
dug into the tile enough to expose the underlying felt 
blanket that is used to mount the tiles to the vehicle’s 
metallic structure. Thus, a small section of the vehicle 
was left without tile protection for atmospheric reentry. 
Measurements and geometry for the divot were 
estimated from the photographs and mathematical and 
physical models of the damage were constructed. 

CFD was used to predict heating for the damaged area 
during entry, a sample of which is shown in Figure 9. 
The methods used to make these predictions had been 
validated for a variety of divot shapes, but this divot 
had a particularly deep, complex, two-tiered geometry, 
putting the heating tool predictions outside their 



Figure 8. Tile damage incurred during the launch of 


STS-118. 





Figure 9. CFD prediction of heating due to STS-118 
tile damage. 

validation range. The heating tools predicted that the 
vehicle could reenter the atmosphere safely with the 
damage, but the uncertainty in the analysis required 
that further testing be performed in NASA’s arc -jet test 
facilities before the vehicle could be cleared for entry. 

A model of the damaged tile system, shown in Figure 
10, was constructed and quickly tested. Data from the 
heating tool were used to set the arc -jet flow conditions 
with significant margin beyond the heating levels 
predicted by the CFD. The conservative test 
conditions showed that the surrounding heatshield tiles 
would be damaged, but would not lose structural or 
thermal integrity. In addition, the exposed portion of 
the underlying metallic structure would not see 
temperatures high enough to damage the vehicles 
primary structure. The CFD analysis coupled with the 
conservative test data provided engineers and managers 
with the confidence to allow the vehicle to reenter the 
atmosphere without the need for a high-risk, high- 
uncertainty repair. Ultimately the Space Shuttle 
Orbiter successfully returned to earth with little 
indication of damage due to excessive heating in this 
area. 

All geometry modeling, model construction, 
analysis and testing for this effort were conducted in a 
matter of days while the vehicle was on orbit. Due to 
the complexity of the damage geometry, the heating 
tool used to predict the reentry temperatures at the 
damage site was used outside its validated range of 
applicability. But even though engineers and 
managers felt it necessary and prudent to conduct 
testing at conservative conditions to ensure the integrity 
of the thermal protection system, the CFD analysis 
played a significant role in the overall decision process 
to reenter the vehicle without repair. The method 
developers received some validation data on this 


geometry from the arc -jet testing prior to reentry, but in 
this case the application of the CFD had clearly begun 
to edge ahead of the method validation. Engineers 
and managers realized the value of the time critical data 
that could be extracted from the methodology even if it 
was applied beyond its validated range. 



Figure 10. Model of STS-118 tile damage for use in arc- 


jet testing. 

STS-124 - Launch Pad 39A Flame Trench Damage 

The launch of STS-124 caused significant damage to 
Kennedy Space Center’s Launch Pad 3 9 A. Shortly 
after SRB ignition, 3500 fire bricks were torn from the 
east wall of the Pad 39 A SRB flame trench and 
exhausted north away from the launch pad, see Figure 
11. Each of these bricks weighed approximately 20 
lbm (9Kg) and brick fragments were clocked using 
radar at up to 1000 ft/sec as they exited the flame 
trench. Beyond the damage to pad facilities, such as 
the flame trench, fencing, and other ground equipment, 
there was concern whether brick fragments, if liberated 
on future flights, could somehow make their way back 
to the Space Shuttle and damage it during its liftoff 
from the pad. 



Figure 1 1. Kennedy Space Center Launch Pad 39 A 
damage after launch of STS-124. 



CFD analysis was used to provide two critical sets 
of data in the subsequent investigation and for the 
flame trench repair. First, it was used to predict the 
time-dependent pressure and temperature environment 
along the walls of the flame trench as depicted in 
Figure 12. Second, it was used to predict the 
flowfield properties within the trench for use in debris 
transport analysis, as seen in Figure 13. This problem 
posed some extreme challenges for the CFD methods 
and those trying to apply them. First, the geometry is 



Figure 12. CFD prediction of the flow environment 
along the walls and floor of the Pad 39A 
flame trench shortly after SRB ignition. 


very complex. The SRBs fire through a pair of holes 
in a part of the pad known as the Mobile Launch 
Platform (MLP). The MLP and the lower portion of 
the Space Shuttle SRBs are shown as the gray 
geometry in Figure 12. The two SRB plumes pass 
through the holes in the MLP and impact on the main 
exhaust flame deflector as depicted in Figure 13. The 
main flame deflector redirects the plume horizontally 
into the flame trench and allows it to exhaust out the 
north side of the pad and away from the launching 
vehicle. The other side of the main exhaust deflector 
redirects the three Space Shuttle Main Engine (SSME) 
plumes to the south and away from the vehicle. 

In addition to the geometric complexity, there is 
significant flow physics complexity as well. The SRB 
plumes involve chemically reacting species exhausted 
at very high speeds and interacting with still air. The 
SRBs generate a severe ignition overpressure transient 
at startup that makes its way down the flame trench and 
is believed to have initiated the separation of the bricks 
from the east wall of the flame trench. Finally, the 
launch process involves the injection of an extremely 
large volume of water into the SRB holes and across 
the top surface of the MLP deck to provide acoustic 
suppression during the launch. 

These physical and geometric features posed a severe 
challenge to the application of CFD, and forced its use 
well beyond the limits of validation and previous 
experience. The analysis was performed in a time 
accurate mode with two gases, one for the SRB plumes 
and the other for the surrounding air. Neither 


chemical reactions in the plumes or any attempt at 
simulating the water deluge were included in the 
analysis. The initial pressure and temperature 
transients predicted along the flame trench wall were 
used to provide structural design data used in the 
subsequent flame trench repair. The steady state flow 
data was used to feed debris transport predictions 
should a similar event occur in the future. 



Figure 13. CFD prediction of SRB plume and associated 
flowfield during launch of STS- 124. 


The design for the flame trench repair was based on 
the CFD loads with significant margin included for 
modeling uncertainty. Similarly, the debris transport 
analysis involved significant deviations from the 
nominal predicted environments in the trench to ensure 
that a conservative estimate of the probability of debris 
striking the vehicle could be obtained. Again, this 
analysis and design was done on a very compressed 
schedule as repairs had to be accomplished prior to the 
next shuttle mission and the vehicle had to be cleared 
for debris impact before it could be launched. In the 
end, a repair could be formulated and applied to the 
launch pad on a schedule that did not impact the next 
launch and the debris transport analysis showed no 
credible mechanism for debris to impact the vehicle. 

The significant point here is that the CFD analysis 
was knowingly applied well beyond its known range of 
applicability, and this was compensated for in the final 
design and debris analysis by adding large uncertainty 
margins to the predicted data. This approach had an 
unexpected beneficial aspect. The high uncertainties 
in the predictions provided motivation to acquire data 
for comparison with the flow predictions to help 
validate them and the designs and analyses they 
influenced. Instrumentation was added to the flame 
trench to measure key environments during the 
subsequent launch. Greater focus was placed on 
obtaining and analyzing debris transport information 
near the pad during subsequent launches as well. 

Thus, when engineers took the risk of applying their 



methods outside their comfortable range of validation 
and generated results with acknowledged high levels of 
uncertainty, they found that they could more effectively 
influence the acquisition of needed validation data for 
their application. 

STS-126 - Flow Control Valve Poppet Failure 

The final Space Shuttle example discussed in this 
paper is the failure of an Orbiter internal component 
that is used to control the pressure in the Liquid 
Hydrogen (LH2) propellant tank during launch. The 
LH2 tank, located in the ET is pressurized using a 
Gaseous Hydrogen (GH2) bleed from the SSMEs. 
Three Flow Control Valves (FCV) control the flow of 
GH2 back into the LH2 tank and thus manage the 
pressure in the tank. During the launch of STS-126, 
one of the FCVs transitioned to a high flow state 
without being commanded to do so and remained in 
that state for the remainder of the flight. Since the 
FCVs are located in the Space Shuttle Orbiter, they 
could be removed and inspected after the mission. 

This inspection revealed that a piece of the poppet on 
the suspect FCV had broken loose during the launch, as 
shown in Figure 14. 



Figure 14. Inspection of SSME FCV after STS-124 


mission reveals broken poppet. 

The potential adverse consequences of this type of 
damage are twofold. First, with the broken poppet, 
the given FCV is free to pressurize the LH2 tank 
continuously and if the break is large enough or 
multiple FCV poppets simultaneously fail, the LH2 
tank could be over-pressurized with catastrophic 
consequences. Second, if the energy and size of the 
poppet fragment is large enough, the fragment could be 
propelled downstream in the pressurization system with 
enough momentum to potentially puncture the GH2 
repressurization piping as it tries to negotiate the 
various turns, bends, and diameter changes along the 
way from the Orbiter to the ET. This piping runs 
through areas in the orbiter, which if subjected to a 
gaseous hydrogen leak, could result in external 
combustion of the hydrogen, again with catastrophic 


results. In fact, borescope inspection of the pipes in the 
Orbiter downstream of the FCV showed several dents 
and scratches that were concluded to be caused by the 
failed poppet fragment as it made its way through the 
pipes. The fragment itself was never found and it was 
assumed to have been transmitted all the way to the ET 
where it was lost after stage separation. 

Figure 15 shows a cutaway schematic of a FCV 
with the GH2 flow path indicated. GH2 enters the 
system from the top at very high pressure and flows 
around the poppet, which is translated left and right in 
the figure to open and close the flow path. The GH2 
then enters the outlet tube and flows though a long 
system of pipes to the LH2 tank in the ET. The 
poppet position is controlled by a solenoid in the FCV. 
When the subject poppet fragment broke off, it created 
an open flow path for GH2 that could not be shut off. 


Orbiter MPS GH2 FCV 



poppet control valve. 


CFD and debris transport analysis were again key 
contributors to the investigation of this problem and the 
risk analysis for future flights that might encounter a 
similar issue. The FCV flow path, including the flow 
through much of the downstream piping was modeled 
using CFD. Both nominal and broken poppet 
configurations were analyzed and steady state and 
transient analyses were performed. Figure 16 shows a 
two-dimensional steady state CFD analysis of the flow 
through a nominal FCV with an intact poppet. The 
stream traces show the complex nature of the predicted 
flowfield downstream of the poppet. There is a very 
high pressure ratio between the flow upstream of the 
poppet and that downstream resulting in a very strong 
jet that emerges from the poppet and impacts the wall 



Figure 16. Two-dimensional steady CFD analysis of the 


nominal FCV operation. 



of the outlet tube. Two regions of recirculating flow 
are generated by this jet, a clearly visible, large region 
below the jet and directly behind the poppet and a 
smaller area in the triangular region above the jet. 

Figure 17 shows a transient analysis of the 
developing flowfield behind a broken poppet just after 
it has fragmented. In this case, the poppet was fully 
closed and there was low flow mass through the poppet 
when it broke. In this case, classical one-dimensional 
flow features such as the propagating shock and contact 
discontinuity can be identified in the more complex 
two-dimensional flow. This type of information was 
used to estimate the downstream acceleration of the 
poppet fragment immediately after it was released from 
the poppet. These data were then input to a modified 
version of the debris transport methodology to predict 
the trajectory and velocity of the fragment as it moved 
downstream. 



Figure 17. Transient CFD analysis of FCV flow just 


after poppet has broken. 

This analysis required that not only the CFD be 
stretched well beyond its validated range of application, 
but also the debris transport methodology used to 
predict the trajectory of the poppet. In fact, new 
debris transport capability for pipe flows with multiple 
impacts was developed on the fly as the analysis 
progressed. Some testing was conducted in parallel 
with the debris trajectory application and development, 
but this testing was very cursory in nature. The debris 
simulations were used to perform a Probability Risk 
Assessment (PRA) for poppet debris released from a 
FCV From the PRA, guidelines were established for 
the maximum size of poppet debris that could be 
tolerated without sustaining collateral damage to the 
downstream pipes. Ultimately the problem was 
solved by requiring highly detailed inspection of the 
poppets prior to each flight, paying particular attention 
to poppets that have been used a high number of times. 

Space Shuttle Lessons Learned 

During the Columbia accident investigation, 
engineers found that the CFD codes provided data that 
appeared to be physically realistic, but they often only 


had limited test data to quantitatively verify their 
answers. Ultimately, many of the analyses performed 
during the accident investigation showed sufficient 
promise that significant efforts were undertaken to 
improve tools in anticipation of broader support of 
future flight operations. This provided the developers 
of these tools an opportunity to take a step back and 
formulate experiments and strategies to further and 
more rigorously validate and calibrate the new tools. 
However, the increased technical insight into post- 
Columbia Space Shuttle flights only generated a 
broader range and greater diversity of problems that 
required innovative use of our CFD methods. In 
addition, these applications were all performed on a 
time-critical basis since they required answers to be 
generated while the vehicle was on orbit, or in the days 
and weeks leading up to a launch. 

NASA engineers sometimes found themselves 
employing their CFD methods to previously 
unanticipated applications with minimal quantitative 
data against which to measure their results. If a given 
on-orbit or preflight problem generated a long-term 
requirement for a future similar capability, then testing 
could be formulated and conducted after the fact to 
help validate the new capability and bring it into the 
stable of tools used to support future flights. If the 
problem was deemed to be a one-of-a-kind application 
with little or no requirement for future application, then 
no data to support the application might ever be 
generated. Regardless of its future applicability, the 
CFD analyses of these time-critical problems were 
often the only data that could be generated on a time- 
scale necessary to address the problem and with 
sufficient accuracy to characterize the issue. Thus 
NASA engineers and researchers sometimes found 
themselves shifting into the “Develop, Apply, Validate” 
paradigm. From a technical rigor standpoint this is a 
less than ideal approach to developing new capability, 
but interestingly, the approach has opened doors for 
developers to acquire data that generally is extremely 
difficult to obtain in the more ideal “Develop, Validate, 
Apply” maturation strategy; namely full-scale 
operational and flight data. 

With the CFD techniques being applied in this 
increasingly aggressive fashion, engineers were forced 
to add generous uncertainty margins to their results to 
account for the fact that they didn’t have sufficient data 
or experience against which to benchmark their 
analyses. It became commonplace to see 20% to 30% 
additional margin added to computational results as a 
“modeling uncertainty factor.” Obviously adding this 
type of margin has operational and vehicle performance 
implications. As a result, Project Managers often 
become more motivated to acquire data to reduce these 
uncertainty margins. Often this data comes from the 
flight system it most directly impacts through 
additional instruments and data systems on the flight 
vehicle. Flight data has always been the gold standard 



for validation of CFD methods, but acquisition of this 
type of data is typically difficult, if not impossible, to 
obtain in a focused, standalone CFD development 
effort. Thus the aggressive application of NASA’s 
CFD methods within the Space Shuttle Program has led 
to an increased number of opportunities to obtain 
highly sought after flight data for method evaluation 
and validation; a turn of events that certainly could not 
have been envisioned a priori. 

NASA’s Constellation Program Further Reinforces 
the Paradigm Shift 

This evolutionary process has been continued with 
NASA’s efforts to provide a broad array of 
aerodynamic and aerothermodynamic data products in 
support of the Agency’s Constellation Program (CxP) 
manned spaceflight initiative. CxP designs have 
pressed engineers to predict the performance of 
aerodynamically complex vehicles, again with minimal 
applicable validation data against which to benchmark 
their methods. The CxP vehicles require analysis of a 
broad spectrum of flow issues, ranging from highly 
separated, bluff body flows to multiple plume jet 
interaction flows. Speed ranges run the gamut from 
the very low velocities of launch pad winds through 
transonic flight during vehicle ascent to hypersonic 
reentering flight. 

The CxP has focused its early efforts on the 
development of two primary components, the Orion 
spacecraft and the Ares I launch vehicle. The Ares I, 
depicted in Figure 18, consists of a Space Shuttle 
heritage solid rocket booster as a first stage and a larger 
diameter liquid rocket second stage. From a CFD 
standpoint, major challenges posed by the Ares I 
include unsteady separated flows generated by the 
diameter change in the first/second stage transition and 
protuberances located at numerous locations on the 
vehicle. These unsteady separated flows are 
particularly important for definition of aeroacoustic and 
buffet environments for structural loads. There are 
also a number of smaller reaction control motors 
incorporated on the vehicle that perform tasks like 
vehicle roll control and stage separation. Since the 
SRB first stage is design to be recovered and 
refurbished for flight, similar to the Space Shuttle, it 
must tumble after stage separation. Otherwise, if it 
were to trim nose or tail first, the recovery parachutes 
would likely fail due to high opening loads. Therefore, 
by definition, the tumbling flight of the separated first 
stage involves significant unsteady separated flow. 

Finally, the Ares I is a very high length to diameter 
ratio vehicle. This presents significant modeling 
problems given the small details, such as protuberances, 
that are important to the vehicle performance and must 
be modeled with sufficient resolution to capture their 
induced aerodynamics. As a result, the Ares I grid 
models are extremely large and costly to analyze. 



Figure 18. Artist concept of the Ares I launch vehicle. 


Ground wind loads are also important to the vehicle as 
it rolls out and sits on the launch pad prior to flight. 

By outward appearance, the Orion spacecraft, shown 
in Figure 19, is similar in shape to NASA’s Apollo 
spacecraft, but it is much larger and heavier. 
Aerothermodynamic issues during reentry dominate the 
CFD analysis of this vehicle. At present, the majority of 
design effort is being spent on earth entry from Low 
Earth Orbit (LEO), as in a mission to the ISS. 

However, the vehicle is also expected to operate 
beyond LEO, where earth entry will be at much higher 
enthalpy. This brings issues for which CFD is not 
highly developed into the analysis space, such as 
radiation effects. Reaction Control System (RCS) jet 
interaction with the surrounding vehicle aerodynamics 
is also important for this vehicle. CFD is not well 



Figure 19. NASA’s Orion spacecraft. 



developed for the hypersonic separated flow jet 
interactions posed by this problem. 

In addition to the basic aerodynamics of the Orion 
spacecraft, Orion designers must also analyze the 
performance of the Orion Launch Abort Vehicle (LAV), 
shown in Figure 20. The LAV is designed to provide 
an emergency crew escape capability in the event of a 
launch vehicle failure. It must be able to be operated 
anywhere in the flight trajectory from sitting stationary 
on the launch pad until shortly after first stage burnout. 
The vehicle is statically unstable and requires an active 
flight control system to maintain vehicle orientation 
throughout its flight. The LAV is powered by a main 
Abort Motor (AM) exhausting from the four large 
nozzles about midway up the forward tower. At the 
forward end of vehicle is the Attitude Control Motor 
(ACM), which is a single solid rocket motor that 
exhausts through eight nozzles arranged 
circumferentially around the top of the tower. Each of 
these eight nozzles can be metered individually to 
actively control the pitch and yaw attitude of the LAV. 
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Figure 20. Orion Launch Abort Vehicle (LAV). 

This control system and arrangement of nozzles has 
proven to be exceptionally difficult to analyze and 
predict vehicle performance. When both the AM and 
ACM are operating together, the ACM plumes can 
interact with the AM plumes as shown in Figure 21. 
The case depicted by the figure shows all eight ACM 
plumes firing simultaneously, but in reality, the ACM 
nozzles can be metered so that individual or select 
groups of nozzles can be operating while the others are 
effectively shut off. In these cases, the ACM jets can 
push on the AM plumes that then push on the back of 
the LAV to generate pitching or yawing moments on 
the vehicle in addition to simply the reactive moments 
due to the thrust of the individual ACM nozzles. These 
interactions are highly nonlinear and dependent on the 
flight conditions, and are particularly severe in the 
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Figure 21. Orion LAV AM/ ACM interaction. 

transonic flight regime. The application of CFD to 
this flow is problematic due to the lack of experience 
and data for these types of interactions. The problem 
is compounded by the fact that wind tunnel testing of 
this type of interaction is very difficult, complicated, 
and expensive and in the end, it is virtually impossible 
to match flight conditions for this type of problem in a 
subscale wind tunnel test. CFD is the engineer’s best 
tool to accurately predict these flows under full-scale 
flight conditions, but with the lack of data and 
experience in this problem, the uncertainties 
surrounding the CFD analyses are large. 

An example of how this issue is challenging the 
Orion LAV developers is presented in Figure 22. This 
figure shows the CFD predicted flowfield for the Orion 
LAV with the main abort motors running and two of the 
ACM nozzles firing downward. At certain conditions 
in the transonic flight regime, the two downward-firing 
ACM jets become left-right asymmetric, even though 
geometrically and mathematically they should remain 
symmetric. This predicted asymmetry greatly reduces 
the pitch effectiveness of the ACM and produces a 
yawing moment on the vehicle when a pure pitch is 
intended from this nozzle firing combination. 
Depending on how the CFD method is initiated and 
executed, the asymmetry can be generated to the right 
or left, and in some cases the flow can remain 
symmetric. 

Code-to-code comparisons show that this 
asymmetry is predicted consistently by all the CFD 
methods applied to the problem to date. Engineers are 
struggling with how to account for this phenomenon in 
the vehicle design. There are questions whether the 
phenomenon will ever surface during normal 
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Figure 22. Orion LAV flow asymmetry predicted by 
CFD. 

operations given the constantly changing flow 
conditions and ACM operating parameters. Even if it 
does occur during flight, the question becomes whether 
it will persist long enough to adversely affect the LAV 
performance. 

Issues like this are having a detrimental impact on 
the vehicle designs as large performance margins must 
be built into the concepts with consequential penalties 
in weight and system complexity. Unlike the design 
and development of NASA’s last manned spacecraft, 
the Space Shuttle, CxP is relying heavily on CFD to 
make design decisions and predict vehicle performance. 
The program is performing considerably less testing 
than during the Apollo and Space Shuttle programs and 
as a result we see engineers again applying large 
modeling uncertainty factors on analysis data that 
impacts vehicle designs. Testing that is being 
performed is serving the dual role of vehicle 
performance evaluation and method validation, but 
there simply isn’t enough testing available to 
adequately validate and calibrate the wide range of 
CFD applications required for this program. In the 
case of the aforementioned ACM asymmetry, it is not 
clear that the phenomenon could be accurately 
predicted or characterized in the wind tunnel. So 
again, NASA is highly motivated to use the flight tests 
they have scheduled in support of the program to also 
acquire data that can be used to reduce the modeling 
uncertainty carried in the development of the vehicle. 

In the last year, CxP has flown three test flights, the 
Max Launch Abort System, the Ares I-X demonstration 
flight and the Pad Abort 1 (PA-1) launch pad abort 
flight. Given the broad use of CFD and other 
analytical methods and the large modeling uncertainties 
associated with these analyses, each of these vehicles 
have been liberally instrumented with aerodynamic and 
aerothermodynamic sensors to capture steady pressure 


data, buffet pressure data, aeroacoustic pressures, and 
heating data, as well as flight telemetry information 
from which flight conditions and integrated force and 
moment data can be derived. So NASA CFD 
application engineers and development researchers 
suddenly have new sources of flight data against which 
to evaluate their capability and methods. Flight data 
can be extremely difficult and rare to acquire in the 
normal develop, validate, apply CFD maturation cycle. 
But through the liberal application of our CFD 
capability, particularly for cases where little or no 
validation data or past experience is available, 
developers can suddenly find themselves in a situation 
where managers are highly motivated to provide 
ground test and flight data that can be used to help 
validate their methods and reduce uncertainties. 

In some areas, engineers and researchers have begun 
to identify Flights-of-Opportunity where existing flight 
projects are evaluated and targeted for their potential to 
produce needed flight data for methodology validation. 
As a result of this type of forward thinking, boundary 
layer transition experiments have recently been 
performed on the Space Shuttle 8,9 as it reenters the 
atmosphere. Instrumentation has also been added to 
the flight manifest of the Mars Science Laboratory 
entry vehicle to measure heatshield performance 
parameters as it enter the Martian atmosphere 10 . 

REQUIREMENTS, PROS, AND CONS OF THE 
“DEVELOP, APPLY, VALIDATE” CFD 
MATURATION STRATEGY 

When the validate and apply processes begin to 
merge and switch, engineers and researchers who have 
been classically known as CFD developers must begin 
to work more closely with those applying the CFD 
methods. In the classical paradigm, a developer took 
his methodology all the way through the validation 
stage and when complete, delivered an analysis 
capability that had been tested for a specific range of 
problems. Those applying the methods knew this 
range of applicability and had confidence, and proof, of 
the methods the ability to predict these specific 
problem classes. When the codes begin to be applied 
outside their known range of validation, engineers look 
to the people most experienced with the method, the 
developers, to help guide them through the application. 
Suddenly developers find themselves answering 
difficult questions about specific problems pertaining to 
specific vehicle designs. 

As a result of this closer interaction, the developer 
finds themselves performing tasks differently than in 
the past. First, they find themselves analyzing and 
understanding more cases than the handful of 
validation cases tested under the classical strategy. 
Vehicle applications typically involve parametric 
analyses of hundreds if not thousands of cases, a 
handful or range of which might generate some poorly 
understood flow phenomenon or unanticipated 


performance. The application engineers thus look to 
the developers to help them determine if the 
methodology is indeed capable of predicting these 
events and if they are physically realistic or an artifice 
of the numerical methodology. 

This forces developers to exercise cases on 
geometries that are often much more complex than the 
typical validation case. To sufficiently refine these 
geometries, the grids are also usually much larger than 
for the typical validation cases analyzed. For instance, 
it is not unusual for grids on the Ares I or Orion LAV to 
exceed 50 million grid points. 

Analyzing data on these more complicated 
geometries and larger grids put a greater burden on post 
processing and interpretation of the data. The 
physical characteristics of typical CFD validation cases 
are often very well known by the development 
community and it is usually apparent whether a given 
methodology is behaving reasonably. This is not the 
case when the applications begin to outpace the 
validation. Physical characteristics of these 
applications are usually not very well understood and it 
often takes a large amount of post processing to cast 
results in a form that best highlights these 
characteristics. 

Finally, as the physical characteristics and the 
performance of the methodology become better 
understood, opportunities for ground and flight testing 
begin to emerge. These tests are typically focused on 
predicting the performance and reducing uncertainty of 
a vehicle or system, but when analysis uncertainties are 
high, they also become opportunities to validate the 
methods used to predict performance. Again, the 
application engineers look to the developers to help 
them define and place instrumentation so as to best 
capture the physical events that are challenging the 
methods. As a result, the developer can also find 
themselves becoming more involved in the tests of the 
project, as opposed to tests formulated specifically to 
validate methodology. 

In the “Develop, Apply, Validate” strategy, the 
developer must become more aligned with the 
application engineers and their problems. There are 
definite pros and cons to this alignment. Many of the 
pros have been highlighted previously in this paper, the 
most prevalent of which is that the developers become 
better aware of the problems their methods are being 
asked to analyze and they receive direct input as to the 
problems that require address for future development. 
Access to validation tests can also be improved. 
Acquiring sufficient test data is always a struggle under 
the classical “Develop, Validate, Apply” strategy; 
acquiring flight data under this paradigm is very rare. 
Also, the methods get to the applications engineers 
faster because they don’t go through the validation 
process before they begin application. 

Many of the cons are fairly obvious. In the 
extreme, methods could be introduced to the 


engineering community before they have undergone 
even the most basic of validation. At best, applying 
methods outside their known validation range is higher 
risk and the associated large uncertainties associated 
with this strategy are warranted. The full 
development cycle for methodology is slower using 
this strategy since significant application of the method 
may have occurred prior to the acquisition of sufficient 
validation data to certify the methodology. The 
development cycle can also become a more organic 
process with it becoming increasingly difficult to define 
the beginning or the end of the cycle. This could be 
seen as either a pro or a con, especially when one 
considers the effort required in advocating and 
initiating a new development campaign; a self- 
sustaining process can be very attractive. 

SUMMARY 

NASA is experiencing an explosive growth in CFD 
applications that is outpacing the validation of their 
methods. Two large reasons for this growth are the 
increased technical insight into Space Shuttle 
operations as precipitated by the Columbia accident 
and the development of new manned spacecraft under 
the Constellation Program. Under these programs 
engineers are applying CFD to problems that were not 
or could not be foreseen in the standard 
development/validation cycle. The high-priority, 
time-critical nature of these analyses do not provide 
sufficient opportunity to validate the methods prior to 
their application. Application of the methods in these 
environments lead to large uncertainties and design 
margins being applied to the vehicles and systems to 
offset the lack of knowledge and experience in the 
specific application of the CFD methodology. These 
margins can have a significant impact on vehicle 
performance and reducing uncertainty becomes a 
critical issue to vehicle operations and designs. To 
reduce uncertainties, ground and flight testing is 
formulated and CFD developers suddenly find 
themselves in a position to acquire the much-needed 
data to validate their methods. Thus while there are 
definite costs to operating in a “Develop, Apply, 
Validate” strategy, there are ancillary benefits as well. 

NASA has naturally evolved into this strategy as a 
result of mission events and mission requirements. It 
is unlikely that anyone would have adopted this 
approach as an a priori strategy, simply because it does 
not fit the well-established logical and systematic 
approach to method development. But now that the 
Agency has refined and exercised this strategy for 
several years and started to realize some of its benefits, 
it warrants a closer look as an alternative approach to 
method development. It can be particularly attractive 
in environments where advocacy for tool development 
can be a challenge. 
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